Skip to content

Unique run_id across manually triggered Dags with schedules#59477

Merged
pierrejeambrun merged 2 commits into
apache:mainfrom
steveahnahn:fix-colliding-run-ids-manual-runs
Jan 13, 2026
Merged

Unique run_id across manually triggered Dags with schedules#59477
pierrejeambrun merged 2 commits into
apache:mainfrom
steveahnahn:fix-colliding-run-ids-manual-runs

Conversation

@steveahnahn

Copy link
Copy Markdown
Contributor

Problem

Manual triggers of scheduled Dags could generate the same run_id repeatedly (often aligned to the schedule boundary). Triggering the same Dag twice would then fail with a 409 Conflict due to a duplicate run_id

Solution

When a manual trigger provides a logical_date, Airflow still infers the correct data_interval for the run, but it no longer rewrites run_after to the scheduled data_interval.end. This keeps manual trigger run_ids based on the actual trigger time making them unique across repeated manual triggers.

  • Added a regression test that creates a scheduled Dag, triggers it twice, and asserts the returned dag_run_id values are different:

Manual testing in breeze triggering a daily scheduled dag via manual trigger runids:
scheduled__2025-12-15T00:00:00+00:00
manual__2025-12-15T04:16:39.584387+00:00
manual__2025-12-15T04:16:50.627593+00:00
manual__2025-12-15T04:18:07.769612+00:00

Related

Fixes #59342

@boring-cyborg boring-cyborg Bot added the area:API Airflow's REST/HTTP API label Dec 15, 2025
@steveahnahn steveahnahn changed the title run id no longer collides in manual runs Unique run_id across manually triggered Dags with schedules Dec 15, 2025
@steveahnahn steveahnahn force-pushed the fix-colliding-run-ids-manual-runs branch from b530a1a to a2f862b Compare December 16, 2025 00:55

@bugraoz93 bugraoz93 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Steve, changes looks good! Maybe we should make run_after unique rather than fallback to logicaldate. I remember this was an intentional change. I maybe missing something though

cc: @uranusjr

@pierrejeambrun pierrejeambrun left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just a question.

Comment thread airflow-core/src/airflow/api_fastapi/core_api/datamodels/dag_run.py

@pierrejeambrun pierrejeambrun left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@pierrejeambrun pierrejeambrun merged commit 13f3e26 into apache:main Jan 13, 2026
126 checks passed
github-actions Bot pushed a commit that referenced this pull request Jan 13, 2026
…es (#59477)

* run id no longer collides in manual runs

* ruff format;
(cherry picked from commit 13f3e26)

Co-authored-by: Steve Ahn <steveahnahn@g.ucla.edu>
@github-actions

Copy link
Copy Markdown
Contributor

Backport successfully created: v3-1-test

Status Branch Result
v3-1-test PR Link

github-actions Bot pushed a commit to aws-mwaa/upstream-to-airflow that referenced this pull request Jan 13, 2026
…es (apache#59477)

* run id no longer collides in manual runs

* ruff format;
(cherry picked from commit 13f3e26)

Co-authored-by: Steve Ahn <steveahnahn@g.ucla.edu>
pierrejeambrun pushed a commit that referenced this pull request Jan 14, 2026
…es (#59477) (#60468)

* run id no longer collides in manual runs

* ruff format;
(cherry picked from commit 13f3e26)

Co-authored-by: Steve Ahn <steveahnahn@g.ucla.edu>
ephraimbuddy pushed a commit that referenced this pull request Jan 14, 2026
…es (#59477) (#60468)

* run id no longer collides in manual runs

* ruff format;
(cherry picked from commit 13f3e26)

Co-authored-by: Steve Ahn <steveahnahn@g.ucla.edu>
ephraimbuddy pushed a commit that referenced this pull request Jan 16, 2026
…es (#59477) (#60468)

* run id no longer collides in manual runs

* ruff format;
(cherry picked from commit 13f3e26)

Co-authored-by: Steve Ahn <steveahnahn@g.ucla.edu>
jason810496 pushed a commit to jason810496/airflow that referenced this pull request Jan 22, 2026
…9477)

* run id no longer collides in manual runs

* ruff format;
jhgoebbert pushed a commit to jhgoebbert/airflow_Owen-CH-Leung that referenced this pull request Feb 8, 2026
…9477)

* run id no longer collides in manual runs

* ruff format;
choo121600 pushed a commit to choo121600/airflow that referenced this pull request Feb 22, 2026
…9477)

* run id no longer collides in manual runs

* ruff format;
Subham-KRLX pushed a commit to Subham-KRLX/airflow that referenced this pull request Mar 4, 2026
…9477)

* run id no longer collides in manual runs

* ruff format;
Ankurdeewan pushed a commit to Ankurdeewan/airflow that referenced this pull request Mar 15, 2026
…9477)

* run id no longer collides in manual runs

* ruff format;
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:API Airflow's REST/HTTP API

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Conflicting manually triggered DAG's run IDs

4 participants